Method and apparatus for reducing inaccuracies when processing 

color data with a tone map 

RELATED APPLICATIONS 

This application is related to the application "Method and apparatus for 
reducing inaccuracies when processing color data with a matrix" that has the H.P. 
docket number 10018165 and was filed on the same day as this application. 

FIELD OF THE INVENTION 

The present invention relates generally to digital nnages and more 
specifically to a method and device for reducing inaccuracies when processing or 
creating color data. 

BACKGROUND OF THE INVENTION 

Digital cameras and scanners create image information by converting light 
into electrical signals using a photo-sensor. Typically the photo-sensors are charged 
coupled devices (CCD). The CCD's typically have 3 color filters arranged over the 
surface to sample the light in three different colors. Typically these are red, green, 
and blue. Scanners typically use an internal light source to illuminate the page. 
Cameras may use only the ambient illumination or they may add illumination to the 
scene using a flash. The electrical signal created by each pixel in the CCD is a 
function of the illumination in the scene, the reflectance of the object in the scene, 
the pass band of the filter over the pixel, and the inherent sensitivity of the CCD to 
light. One of the goals of these systems is to reproduce the color of an object on a 
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computer display and with printed output. The computer display typically has 
different output characteristics than printers. One method used to help match the 
display with printers is call tone mapping. US Patent number 5,719,965 entitled 
"Synthesized higher precision look-up table from lower-precision look-up table" 

5 which is hereby incorporated by reference, describes how tone maps are used. One 
type of adjustment done with tone maps is "gamma correction". Ganmia correction 
typically maps reflectance space with a power curve. This curve has the property of 
increasing the contrast of the image in the dark areas and decreasing the contrast in 
the light areas. The tone map operation can be done in the scanner or camera 

10 hardware, or the raw color data may be sent to a computer where the image 
processing (including the tone map operation) is done. 

CCD's sample light and convert it into electrical signals. These electrical 
signals are then converted into digital information typically using an A-to-D 
converter. CCD's are typically fairly linear in response to the intensity of light 

15 falling on each pixel. The human eye is not linear in response to intensity. The 
human eye responds more to density, which is the log of intensity. Because of this 
phenomenon a given change in intensity will be much more noticeable to the human 
eye if the change occurs in a dark area as compared to a light area of an image. For 
example in a system with 8 bits of resolution or 256 levels of intensity and 0 as 

20 black and 255 as white, a 2-count change in intensity from 10 to 12 may be much 
more noticeable to the himian eye than the same 2-count change in intensity from 
200 to 202. 
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The number of levels of light that the CCD and A-to-D converters can create 
has increased in recent years. Older scanners and cameras typically had 8 bits per 
color (256 levels) for a total of 24 bits. Today CCDs and A-to-D converters create 
10, 12 and even 16 bits per color. In addition to the increase in the number of 
5 levels, currently available CCDs have improved their signal-to-noise ratios. This 
means that more of the total number of levels are actual signal instead of noise. 
These changes have made reproductions of the dark areas of the images much more 
accurate. 

Unfortunately these changes have also created some problems. One of these 
10 problems is the tone map operation. In the dark areas of the image, inaccuracies in 
the tone map operation may show up as slight unintentional color changes in the 
image. These inaccuracies in the dark area are exaggerated when the tone map is 
using a gamma correction curve. The human eye may be unable to detect a slight 
change in brightness, but the human eye is very sensitive to small color shifts in the 
15 dark areas of an image. These small changes m color in the dark area of images can 
reduce the usability of these images. 

What is needed is a method and apparatus that reduces the inaccuracies in the 
tone map operation in the dark areas of the image. 

20 SUMMARY OF THE INVENTION 

A method and device for improving the tone map operation in dark areas of 
an image. 
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Other aspects and advantages of the present invention will become apparent 
from the following detailed description, taken in conjunction with the accompanying 
drawings, illustrating by way of example the principles of the invention. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Figure 1 is a chart of a gamma correction curve loaded into a tone map. 

Figure 2 is a flow chart for the tone map operation in accordance with one 
embodiment of the current invention. 

Figure 3 is a flow chart for the tone map operation in accordance with another 
embodiment of the current invention. 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT 

A method and apparatus that reduces the inaccuracies in the tone map 
operation in the dark areas of the image can increase the number of usable images. 

The raw color components of a sample or pixel from the photo-sensor of 
scanners, cameras, or the like are typically adjusted with a tone map operation to 
produce a more usable output color. The tone map operation creates new output 
color components for each pixel by mapping the raw color components from the 
pixel. Typically each color component has its own tone map. For example the raw 
red CCD output would be mapped using the red tone map to produce an adjusted 
red color for each pixel in the image. The tone map typically gamma corrects the 
raw CCD output by decreasing the contrast in the light areas of the image and 
increasing the contrast in the dark areas of the image (see figure 1). 
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When the pixel or sample is from a dark area in the image, non-linearity's in 
the tone map operation may cause unacceptable color shifts for the pixel. The image 
can be improved by avoiding the color shifts in the dark areas of the image by 
selectively applying the tone map operation only to pixels that are not in a dark area 
of the image (see figure 2). The selective tone map operation can be performed in 
the hardware of the scanner, camera, color copier, or the like, or the raw color data 
can be transferred to a computer where the selective tone map operation can be 
performed. 

There are numerous methods to determine if the raw color component is in a 
dark area of an image. In one embodiment a threshold is defined and if the raw 
color component is greater than the threshold the color component is considered to 
be in a lighter area of the image. In a preferred embodiment the threshold level or 
value is approximately 20 eight-bit counts and in another preferred embodiment the 
threshold level or value is approximately 10 eight-bit counts. 

In another embodunent there may be a threshold for each raw color 
contained in a pixel. For example in a pixel that contains 3 color components (red, 
green, and blue) there may be a red threshold, a green threshold, and a blue 
threshold. Each color component would be compared to its own threshold to 
determine if the tone map operation should be performed on that color component. 

Once a color component has been classified as being in a dark area of the 
image, the color component can be preserved. In one embodiment the tone map step 
is skipped leaving the dark areas unchanged from the raw color photo-sensor output. 
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In another embodiment a one to one mapping is used to replace the gamma curve in 
the tone map below the threshold level. 

A sharp transition between output pixels that are tone mapped and output 
pixels that are in the dark areas may create image artifacts. It may be desirable to 
blend or smooth the transition between the dark area pixels and the pixels modified 
by the tone map (see figure 3). When a color component is in a light area of the 
image (304) the tone map is used (306). If the color component is in a transition 
area (308) the color component may be blended or smoothed (310). When the color 
component is in a dark area of the image the raw color component may be used 
(312). There are many ways to define which color components are in a transition 
area. 

In one embodiment two thresholds are defined. In a preferred embodiment 
the upper threshold is 30 and the lower threshold is 10. Any pixel that has a color 
component above the higher threshold is not considered to be in the dark area. 
These pixels are transformed using the tone map. Each color component lower than 
the second threshold is left unmodified. Each color component below the higher 
threshold and between the two thresholds is considered to be in the transition area. 
More than two thresholds can be used to create more than one transition area. Each 
color component falling into each transition area could be treated in a different way. 

Once a color component is defined as being in a transition area the color 
component can be modified to help blend or smooth the transition between color 
component in the dark areas and color component not in the dark areas of the 
image. There are many ways to blend or smooth the transition area color 
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components. In one embodiment the color components in the transition area are 
modified by interpolation between the raw values and the transformed values from 
the tone map. For example if the high threshold is 12 out of 256 levels and the low 
threshold is 8, and a color component has a raw red value of 1 1, and the tone 

5 mapped value is 9.5 then the interpolated value would be between 9.5 and 11. The 
interpolated red value between the raw red value of 1 1 and the tone mapped red 
value of 9.5 would be: (12-8)/(ll-8) = (9.5-ll)/(X-ll). Solving for X gives 9.875. 

The foregoing description of the present invention has been presented for 
purposes of illustration and description. It is not intended to be exhaustive or to limit 

10 the invention to the precise form disclosed, and other modifications and variations 
may be possible in light of the above teachings. The embodiment was chosen and 
described in order to clearly explain the principles of the invention and its practical 
application to thereby enable others skilled in the art to utilize the invention in 
various embodiments and various modifications as are suited to the particular use 

15 contemplated. It is intended that the appended claims be construed to include other 
alternative embodiments of the invention except insofar as limited by the prior art. 
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